Subscribe
Share
Share
Embed
Sounds of Science is a monthly podcast about beginnings: how a molecule becomes a drug, how a rodent elucidates a disease pathway, how a horseshoe crab morphs into an infection fighter. The podcast is produced by Eureka, the scientific blog of Charles River, a contract research organization for drug discovery and development. Tune in and begin the journey.
Mary Parker:
I'm Mary Parker, and welcome to this episode of Eureka's Sounds of Science. Both gene therapy and artificial intelligence seem like science fiction concepts at times. But what happens when you put them together? I'm joined today by Alex Sargent, Director of Process Development at Charles River in our Cell and Gene Therapy Department. Alex and his girlfriend developed an AI algorithm to help wade through the giant piles of data generated in his work, which is something that AI excels at if you do it right. Welcome, Sarge.
Alex Sargent:
Thank you, Mary. Wonderful to join you. I should probably also mention my girlfriend's name is Dana. She'll listen to the podcast and she'll want me to say that.
Mary Parker:
Excellent. Well, thank you, Dana. So can we start with your background? What drew you to science and this field in particular?
Alex Sargent:
Certainly. I grew up in Cleveland, Ohio, Industrial City. Got my PhD in neuroscience, neuroimmunology from Case Western Reserve University. After there I was in academia working on clinical trials in the stem cell fields, looking at stem cells as a way to tackle disease. And when I was in academia, I realized that really a lot of the exciting science in the space of cell and gene therapy that was just emerging.
So I transitioned to a pharmaceutical company. And over the past 10 years have been at different companies, always looking at cell and gene therapies to treat disease. What drew me to this space is from a very early age, I think my primary goal was to cure disease. Doesn't everyone want to do that?
Mary Parker:
Yeah, I think so.
Alex Sargent:
And in today's age of medicine with what we're doing, harnessing the power of the cell or the power of the gene seemed like the best way to have a chance to do that, or at least be a part of a team of smart people to do that. So I consider myself very lucky to have the opportunity here at Charles River to work on some truly exciting and breakthrough therapies every day.
Mary Parker:
I have it on good authority that Cleveland rocks. Is that true?
Alex Sargent:
Cleveland does in fact rock and shout out to Drew Carey in case he's also listening.
Mary Parker:
Oh, I'm sure he is. I'm sure he's a frequent listener. So this is also a bit off-topic. You mentioned wanting to cure diseases. But just out of curiosity, if you could cure one disease right now, what would it be?
Alex Sargent:
It'd have to be cancer. I like just about everybody listening have lost loved ones or know somebody who's personally affected by cancer. It's the emperor of all maladies as it's been named. And I do say day-to-day, I probably spend about 80% of my time thinking of new and better ways to fight cancer.
Mary Parker:
Yeah, that's awesome. I actually had one of the earlier episodes of this podcast I had my friend Carmen on who was a cancer patient. She came on and just gave a very frank appraisal of what it's like to be a cancer patient, especially a known terminal cancer patient. And then she unfortunately died during COVID, but she really wanted to come on and share her story and how important it was to her to be involved in clinical trials. Even though she knew that they couldn't do anything for her, she wanted to leave that piece of data behind for the future. And she did.
Alex Sargent:
People like Carmen who participate in those trials and who help us with our research, they truly are heroes.
Mary Parker:
Yeah, absolutely.
Alex Sargent:
And that is not an overstatement at all.
Mary Parker:
So what does your day-to-day look like at work?
Alex Sargent:
Unfortunately, more of it is spent in meetings.
Mary Parker:
Yeah, same.
Alex Sargent:
Which is natural. But on the good days or the days I'm able to really do what I love, it's spent in the laboratory. I have a wonderful dedicated team where we spend a lot of the time in the laboratory asking questions about how do we make better cellular therapies and how do we make better gene therapies to treat a wide variety of diseases, cancer, neurological diseases, rare diseases affecting children.
The process of how you create those therapies, it's really important to get that right because that informs how effective, how safe, and how potent the therapy can be so it has the best chance of tackling and perhaps even curing that disease.
Mary Parker:
And what are some of the pain points in your work that made you decide that AI was the answer? I'm assuming it was a giant pile of data that you had to sift through manually.
Alex Sargent:
You assume correct. So in today's molecular and genetic age, we live with a lot of data from our cell phones to the Human Genome Project and able to map whole genomes of patients and of individuals. As a scientist, both myself and my team, sifting through all of that data, making sense of all that data we receive or we generate in the lab can be enormously challenging.
Anybody who's gone this route can recall days of graduate school or some other job of just literally mountains of paperwork and mountains of numbers to crunch and Excel sheets to stare at. So we were really interested in developing AI and machine learning as a tool, a powerful tool to help us look at the data, analyze the data, and discover trends, discover interactions that could lead us to use the data to come up with better solutions for what we are trying to do.
Mary Parker:
I think one thing that maybe people who aren't familiar with computers like myself get wrong about AI is the idea that it's replacing a human scientist, which is really not the case at all. It's just replacing the hours of tedious work that you used to have to do manually, and it's helping point you at things that are important that you might've overlooked if your brain was too fried from staring at Excel sheets all day.
Alex Sargent:
Precisely. My team is a big proponent of AI because they have seen firsthand how the hours and days spent on what can be mind-numbing tasks can be replaced with these machine learning and these AI tools. So it really frees up the scientists to tackle harder, more complex questions, gives them more time, while the machines can really analyze the data and help us avoid some of the mundane tasks.
Mary Parker:
So you mentioned when we spoke before about using a mini bioreactor to test and refine your algorithm. How did that work?
Alex Sargent:
Yes, yes. So we have a mini bioreactor platform in our lab and it's fully automated. What I mean by that is viewers might've seen in car factories or automobile assembly lines or other videos the big arms, the automated arms. So we have one of those in the lab. And what that arm does is it takes care of hundreds of little bioreactors, which are essentially little dishes, little vessels that we grow cells in or that we make gene therapies in.
And so the arm can help replace the scientists because cells can be quite needy. They're living things. They need to be fed. They need to be monitored. They need to be, right, carefully kept like plants or even like children, I'm sure some people I work with would say. And so the arm is able to do all that in real time.
And it also has integrated sensor technology so that we can record from each of these little bioreactors, each of these little cultures a lot of data about how they're doing, what kind of things are they producing, how are they growing, how are they responding, say if we challenge them with an environment like happens in cancer and a tumor, or if we actually challenge them with cancer cells themselves.
And we're able to collect all of that data and design really large experiments leveraging the power of robotics. So you don't need 10 people and they're constantly taking care of everything. And then we have all that data, and the challenge is how do we analyze it, how do we interpret it, what do we do with it?
Mary Parker:
I'm imagining this robot arm and just thinking about how it's good not to give robots legs because then they can't chase you down. Not that I'm afraid of robots. Don't tell robots I said that.
Alex Sargent:
I will not tell the robot. We call her Amber and she's quite a good robot, so I think you'd like her.
Mary Parker:
I love good robots. So can we talk about the importance of properly entering data for any algorithm to work? This is another thing that I think people think you can just throw in any data in any format. But no, that is not true. Can you tell us about that?
Alex Sargent:
Yes. One of the first rules I learned about AI or machine learning was that the algorithm you develop, the AI model is only as good as the data you use to train it, the data you use to really build it. And an excellent example of that, this was told to me by a colleague some years ago when I was getting into AI, was a study out of the University of California Irvine, where they were looking at building an AI algorithm to distinguish between wolves and dogs.
Now, you might ask, why would we need a program to do that? That actually came up for ecology research. So ecologists, people who study wild wolf populations, they would set out cameras with bait or attractants for the wolves and take pictures of the wolves to understand where the wolves are, how close are they getting to civilization, where are they migrating or traveling, some really important work.
The problem is, as you can imagine, leaving out mounds of meat or whatever bait they were using also attracted dogs, domestic dogs, dogs on a hike with their owners, wild dogs. And so it was the task of graduate students to sift through literally tens of thousands of pictures and say, "Oh no, that's a dog. We don't care about that one. Or, oh, that's a wolf. Yeah, we actually need to flag that picture so we can follow this wolf."
So the researcher said, "I'm going to create an AI algorithm that distinguished between the two and save that poor graduate student a lot of time." What they did is they uploaded pictures of wolves and dogs from the internet, from good old Google images, thousands, tens of thousands of pictures to train the algorithm, this is what a wolf looks like, this is what a dog looks like.
And after completing that training exercise and testing the algorithm with pictures from the internet, the algorithm was very successful at saying, "Nope, that's a wolf. Nope, that's a dog." When they deployed that algorithm in the field, when they used real pictures that the cameras out in the wild were taking, they found that the algorithm failed miserably. Every single picture that it was getting, it was saying, "No, no, no, that's a wolf," even though there were tons of dogs.
And what they realized is that the algorithm didn't learn how to distinguish between a wolf and a dog. An algorithm learned to distinguish between the backgrounds. The pictures of the wolves on the internet were often in snowy environments with a lot of white. They were almost always in forest or wild backgrounds. And the pictures of dogs were in yards, were in houses, in cities and scapes like that.
So when it came time to put the rubber to the road and really test this algorithm where it was supposed to be, it saw the pictures of the forest and the trees and the snow and the wild background and it said, "Oh, I know that background. That's the background that I should say is a wolf." So it really illustrates two things. One, like I mentioned, the training data can create very strong biases in your algorithm, and your algorithm is only as good as the data you use to create it.
And the other thing that's important in this scenario is to realize that these algorithms are somewhat of a black box. The model can't tell you why it does something. It can't say to the operator, "Well, yeah, of course, I flagged that as a wolf. Look, there's trees in the background." So it actually took a lot of time and attention to figure out what the algorithm was picking up on in order to figure out why it was failing at the task it was assigned.
Mary Parker:
It's also illustrative of why humans are still important in the process. Because even though I suspect you can eventually train an AI to distinguish putting a dog and a wolf, you might not necessarily get the subtle distinction between a wolf and a husky, for example, especially on something like a trail camera, which is not the highest resolution camera typically.
Alex Sargent:
Exactly.
Mary Parker:
So it'd still be like, "That's a wolf," but you need a human to look at that and be like, "No, that's a Malamute. Good try though, computer."
Alex Sargent:
In our own experiments in the labs and building our own algorithms, we learned that, yeah, the computer, the machine learning program would make mistakes. It would tell us things that we could clearly recognize as wrong or, well, this doesn't make sense.
And yeah, you needed that person there who understood the biology to understood what's actually happening here to say, "No, I think the direction the algorithm's pointing us towards is not correct," and to be able to take some of the other trends or the other things that it identified and say, "That looks more promising. Let's investigate that more."
Mary Parker:
Can you set a threshold for the data? I'm thinking specifically of something like training it to look at a million different mammograms and being able to recognize ones that should be looked at by an expert versus ones that are definitely clear. And that margin for error could be quite wide, leaving the doctor looking at a lot of images that are perfectly fine, but it still could cut out hundreds of images that they probably don't need to look at too closely and thus saving them time.
Alex Sargent:
Yes. You bring up a great point. A friend of mine recently had a mammogram and was quite annoyed because she had to go to the doctor for some follow-up because she was flagged as potential positive. The doctor said, "Oh, no. After doing some initial review, you're quite okay and healthy. It's that darn AI algorithm we're using. It flagged you as a positive because we said it so that it is a little liberal on what it's going to define as something to be concerned about."
Mary Parker:
As it should. As it should.
Alex Sargent:
As it should. As it should. So with these algorithms, you can really help set and define how conservative it should be in its decision-making, how adventurous or liberal it should be in its decision-making, and that depends on a case-by-case basis for what you're building the algorithm. For detecting cancer, yes, I would want the algorithm to err on the side of the caution.
Mary Parker:
Absolutely.
Alex Sargent:
And say, "This is cancer," bearing in mind that it can't go too far or otherwise it really loses its utility as a tool if it just pulls everything cancer.
Mary Parker:
Yeah, absolutely.
Alex Sargent:
On the other side of things, if you wanted something to be more conservative, a classic example for theirs in drug discovery, we're testing thousands of different compounds to see which one has an effect. A lot of those compounds might have a very small effect, but we're training the algorithm that, no, we need to see a 10, 100, 1,000-fold increase consistently, and then you flag that for us and report that back to us. So it all really depends on what you're designing the algorithm to do.
Mary Parker:
Yeah, that's a good way to think of it. Can you give me an example of one certain test that you do in cell and gene therapy that you can use the AI for a specific scientific test?
Alex Sargent:
Right, right. So one specific test comes along for microbial detection.
Mary Parker:
Oh, of course.
Alex Sargent:
Right, right, right, right. So how that works is you take a small sample from your drug product, your cell or gene therapy, and you put it on a plate. And if there's bacteria or something in that plate, in that system, they should grow and you can add a reagent where it should turn color. Now, as you can imagine, before automation and AI, then you would have somebody screening through thousands of these plates and seeing, "Ah, this one didn't turn color. It's safe. This one did turn color. Oh, it's contaminated."
And so now setting up an AI system where you're taking thousands of pictures of the plates, that's going into an algorithm and it's able to make the determination, yes, that changed color, there seems to be a contamination event. No, that didn't change color, that's sterile, that's safe to release to the patient, for instance.
Mary Parker:
I think that color is actually something that AI is really good at. Because if you think about it, let's say an extremely specific and discrete computer data point, like how many pixels of RGB you have in a certain frame, so yeah, that makes sense that it would be good at that.
Alex Sargent:
Precisely. And back to our previous point, what kind of questions are good for AI? Very black and white literally in this case.
Mary Parker:
Yes, no.
Alex Sargent:
Yes, no. It's this color or it's not this color.
Mary Parker:
And this might not be the case for this particular test, but if it was, imagine if it just changed color a little bit, just so enough that a human might not notice, but a computer would absolutely pick on even the slightest change in yellow.
Alex Sargent:
Right, right. And that's one of the scenarios you can imagine like they talked about where you'd want to set the threshold to be fairly conservative of what it defines a color change. So even the lightest possibility, it's going to say, "Yes, I think there's something here," so you don't have a rate of too many false negatives.
Mary Parker:
Of course, on the other hand, you'd still need to make sure that you're using basically the same light bulb to light the sample each time. Otherwise, it'll think that it's changed color when it's really just daylight instead of pure white light bulb.
Alex Sargent:
Precisely. And you still need the person because if the AI flags it and says, "This changed color," you still need a set of eyes to look at it and confirm that, "Yes, I agree. This has changed color. We can't release this."
Mary Parker:
Yeah, perfect. So speaking in that same vein, what might humans miss if they're looking at a giant data set without the help of AI? You're getting into that zone where your eyes start glazing over and you just don't really look closely at the data in front of you.
Alex Sargent:
I think one of the biggest advantages of AI when looking at data, as we already mentioned, large data sets. So for something small, AI might not be well suited for that in part because it takes a very large data set to train an AI algorithm, to develop and build a good AI algorithm. So the first criteria is that you have something where you have a lot of data that's enough to train in AI, but also might be a little too daunting or too much for a human to tackle.
The other thing is you have to think very carefully about the type of question you're asking, and can AI, can a system actually answer this question? A good example here is when you're thinking about image analysis, looking at pictures. Everyone's familiar with this AI asking a question, can you distinguish between a dog versus a wolf? And if they are dissimilar enough, then yes, it does make sense that an AI or a system we can train and develop to do that.
But taking that same scenario, if you ask the AI, can you distinguish between this dog and can you distinguish between this dog when it's sad or when it's scared, picking up emotional cues, and AI has no concept of emotion. We defined what scared means. We defined what sad means. So that's likely going to be very challenging for an AI. Perhaps not impossible, but a lot more challenging.
And that may be a question better left to a human or to an observer. Things that are subjective and very much human in nature tend to lend themselves better to humans to qualify and to analyze, as you can imagine.
Mary Parker:
Yeah, absolutely. But I mean, there is some we can do with that. We did a story a while ago about there was an AI analysis of a bunch of data gathered from video cameras that were trained on lab mice. And the cameras and the AI could pick up, for example, when the mouse was not running back and forth 15 times a day, but instead 13 times a day, things like that.
It was like, oh, the mouse was 2% less active today than it was yesterday. It doesn't know what the mouse is feeling. All it knows is that it moved 2% less on Thursday than it did on Monday. And then the human comes in and is like, okay, well, it's having an effect, or it's sick, or it's whatever.
Alex Sargent:
Precisely. Precisely. You're looking at things that are very quantitative and relatively objective as opposed to subjective. That's really best for the kinds of data and the kinds of obstacles we want to tackle with AI and machine learning.
Mary Parker:
So we might've already covered this, but what does AI miss that we still need humans to correct?
Alex Sargent:
So that's a very important topic of discussion right now in the field of cell and gene therapy. So in CGT, my team, for instance, and others, we're really using AI and these machine learning tools in the research and development side, development side of coming up with better ways, better systems, and analyzing data to help us do that. The other side of that is the manufacturing side. So just as we manufacture cars in an assembly line, we manufacture these therapies using robotics or using people oftentimes on the spot de novo for that patient.
And here it's very tenuous of can we use AI and should we use AI? An example, let's say you're making a cell therapy. That's going to generate hundreds, if not thousands of pages of documents detailing meticulously how that therapy was made, which step, an operator or machine added so much of this reagent, took out so much of that, all sorts of different data. And right now that data is meticulously reviewed by a quality review team, a quality assurance team, step-by-step scanning for errors, scanning for inconsistencies before we release that product to the patient.
And that's for the benefit of the patient, both in terms of efficacy, that we're making an effective therapy, but also safety, that something didn't go wrong that could harm the patient while we were manufacturing this therapy. There is some talk about using AI to scan those hundreds or thousands of pages of documents, doing something, for instance, as it's scanning, just looking at the calculations that have been performed and are there any errors, or looking at typos, things that are a bit unclear or not written down correctly or recorded correctly using the electronic system.
And while that can be good, we have to remember at the end of the day, a person has to sign off on everything. A person has to look through all of that and say, "Yes, I verified the process. I verified everything here. This is safe. This has been performed correctly and can go to a patient." Because at the end of the day, the regulatory agencies, the Food and Drug Administration, for instance, they can't hold a machine accountable. They hold us as the manufacturer accountable.
So where AI and machine learning might be very good on the R&D side, I would argue that on the manufacturing side, it's still very much people driven from a quality control perspective. And we have to be very careful about our implementation of AI tools to bypass people when it comes to actually making these therapies.
Mary Parker:
Interesting. Do you think it would be possible to have the human QC and the AI QC running simultaneously for let's say a year, two years, and compare their results? Go with the human and trust the human, but then look back and see where they might've met up and where AI might've failed.
Alex Sargent:
Yes, yes. And that I think is very critical to successfully rolling out AI tools is always safeguarding and making sure you're comparing it to the other systems, the human, the manual system, to make sure it's performing as well. And really it needs to be better for it to make sense to use. Even when you have that data though, it's important to continually monitor, and it's important to be careful about the use of AI. And one of the reasons is it's in cognitive science, it's called a fallacy, the illusion of objectivity.
Especially when it comes to AI, we think AI is being entirely objective, not having a bias or a viewpoint. But in fact, the dog versus wolf scenario illustrates, you can get biases that do develop in the AI algorithms. So if you have an AI algorithm in the QC environment and for three years it's analyzing all this QC data, it's learning, remember, and it can develop new biases by looking at real world patient samples, that it then projects into all of the things that it's analyzing for the next several years.
So you have to continually monitor that and make sure that you're not building bias in your AI. Another risk that comes with AI is this idea of explanatory depth. We rely too much on the algorithms, and people tend to review things less carefully because they say, "Ah, the AI is doing that." So you test the AI system. It works. You implement the AI system. Now you find that your human reviewers that you're using with the AI, they become a little sloppy. They look at the documents or things a little less carefully because they say, "Oh, I got the AI. That catches everything."
That is a real risk in any industry, but especially when we talk about making medicines. And so we also have to take safeguards to make sure that that doesn't happen as well.
Mary Parker:
Yeah, no, that makes perfect sense. Well, thank you so much, Alex, for sharing your expertise with me. It's been a treat to talk to you.
Alex Sargent:
Yes. It had been a wonderful conversation. Thank you so much. And a quick shout out, thank you to Dana as well for helping me.
Mary Parker:
Thank you, Dana.
Alex Sargent:
Yes, yes, yes. And I would say one last thing, go Cleveland Guardians.